Papers with cross-attend representations of intermediate encoder activations
DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing interpretability methods have been proposed to interpret the inner workings of Transformer models at different levels of precision and complexity. |
| Approach: | They propose a method to analyze encoder-decoder Transformers by using the decoder module Model Output encoder to cross-attend representations of intermediate encoder activations instead of using the default output. |
| Outcome: | The proposed method maps uninterpretable representations to human-interpreted sequences of words or symbols, shedding new light on the information flow in this popular but understudied class of models. |